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ABSTRACT 

We study the dominant central giant elliptical galaxies in "Fossil groups" using deep op- 
tical (R-band) and near infrared (Ks-band) photometry. These galaxies are as luminous as the 
brightest cluster galaxies (BCGs), raising immediate interest in their link to the formation of 
BCGs and galaxy clusters. However, despite apparent similarities, the dominant fossil galax- 
ies show non-boxy isophotes, in contrast to the most luminous BCGs. This study suggests 
that the structure of the brightest group galaxies produced in fossil groups are systematically 
different to the majority of BCGs. If the fossils do indeed form from the merger of major 
galaxies including late-types within a group, then their disky nature is consistent with the re- 
sults of recent numerical simulations of semi-analytical models which suggest that gas rich 
mergers result in disky isophote ellipticals. 

We show that fossils form a homogeneous population in which the velocity dispersion of 
the fossil group is tightly correlated with the luminosity of the dominant elliptical galaxy. This 
supports the scenario in which the giant elliptical galaxies in fossils can grow to the size and 
luminosity of BCGs in a group environment. However, the boxy structure of luminous BCGs 
indicate that they are either not formed as fossils, or have undergone later gas-free mergers 
within the cluster environment. 
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1 INTRODUCTION 

It is believed that most of the large and luminous ellip tical galax- 
ies have formed via mergers of disk ga laxies iToomre & To omre 
1 1 9721 : Isearle. Sargent & Bagnuololll973|). This h as been suggested 
by morphology density relation l(DressleJl98Ch . the observed fre- 
quency of merging galax ies at high red shift and also extensively 
in computer simulations (Barnes 1989). Many of these luminous 
ellipticals (Mb < -21) are found in rich galaxy clusters. This, 
however, does not imply that they are formed in cluster environ- 
ment. The brightest cluster galaxies (BCGs) are of special inter- 
est as they reside close to the centroid of cluster X-ray emission 
bones & Formarll984f) . and the centre of the dark m atter distribu- 
tion i n clusters, as inferred from gravitational lensing ISmith et al"l 
2005) - implying that they lie at the minimum of the cluster poten- 
tial well. They also show various correlations with cluster proper- 
ties. 

In general, two main formation modes could be assumed for 
the hierarchical formation of BCGs according to their formation 
environment: 1) BCGs formed in the high velocity environment of 
clusters. Although the effectiveness of dynamical friction in bring- 
ing individual galaxies to the cluster centre via orbital decay will 
be reduced by the high velocity dispersion, infalling groups will 
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still suffer rapid orbital decay if they survive long enough, and can 
then deposit their brightest galaxies in the clust er core, where they 
can merge and form a bri ght elliptical galaxy iLin & M ohi 2004; 
Hausman & Ostrikei 1978). 2) BCGs are formed in the low ve- 
locity environment of groups, where dynamical friction causes the 
orbits of individual galaxies to decay, resulting in the merger of 
all large galaxies, if the group forms early and is left un disturbed 
for a sufficiently long period (Dubinski 1998; Ponman et al. 1994; 
Jone s et ai"ll2003h . The group containing this 'ready made' BCG 
then provides the nucleus around which a cluster forms. 

Elliptical galaxies show fine structures and are more complex 
than originally thought. These structures take the form of hidden 
disks, shells a nd bars, departures from pure elliptical isophotes 
feendei|| l988|) and variati ons in radial surface brightness profiles 
( Khosroshahi et al. 2000), some of which are found to be envi- 
ronment dependent iKhosrosh ahi et al. 120 04). BCGs formed in the 
above two modes should display several observational signatures of 
how they formed. A useful probe is provided by galaxy morphol- 
ogy - isophotal shapes, radial surface brightness profiles and the 
presence or not of multiple-nuclei. Different star forming histories 
are also expected. Some of these studies require space resolution 
data, but some can be studied using ground based observations, in- 
cluding the isophotes of elliptical galaxies. 

Based on their isopho tal shapes, el liptical galaxies can be 
classified as disky or boxy jBendeJ fl988). Low mass ellipticals, 
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which are fast rotating, are usually disky isophote galaxies with 
positive fourth-order Fourier coefficient B4 > 0. Some disky- 
isophote elliptical s migh t contain faint disks similar to SO galaxies 

1 Scor za & B ended fl995h . With negative B4, boxy isophote ellip- 
ticals are less rotationally supported. They g enerally contain flat 
cores iFaber et all 1 1 997|jLameet_ al. I 12 003 ) and show complex 
internal kinema tics I Em sellem et al. Il2004h . The observations of 
iRest et all 1200 lh show that it is very unlikely to find disky ellip- 
ticals which are also cor e galaxies. The majority of the BCGs are 
found to be core galaxies The distinct observed 
properties of disky and boxy isophotes of elliptical galaxies, such 
as their radio properties show that they are more than just an arti- 
fact o f viewing angle or the projection on the plane of sky iB ended 
Il989h . 

It is important to understand the origin of isophotal shapes be- 
fore they can be used t o trace the formation history of ellipticals. 
Naab & Burkert (2003) performed a large survey of dissipation- 
less merger simulations of disk galaxies and found that unequal- 
mass 3:1 to 4:1 mergers lead to fast rotating disky ellipticals while 
equal-mass 1:1 to 2:1 mergers produce slowly rotating, p ressure- 
supported ellipticals. However ^Khochfar & Burkerth2005l) showed 
that the above scenario is not able to reproduce the observation that 
the fraction of boxy and disky ellipticals depends on galaxy lumi- 
nosity. They argued that equal-mass mergers lead to boxy ellipti- 
cals and unequal-mass mergers produce disky ellipticals. However, 
major mergers between bulge-dominated galaxies result in boxy 
ellipticals, independent of the mass ratio, while merger remnants 
that subsequently accrete gas, leading to a secondary stellar disk 
with more than 20 per cent of the total stellar fraction, are always 
disky. More recently they showed that mergers of spiral galaxies 
alone cannot reproduce the kine matic and photometric properties o f 
very massive elliptical galaxies iNaab. K hochfar & Burkerl2 006). 
nor can they reproduce the observed correlation between isophotal 
shapes and the luminosity of ellipticals. 

Here we study the isophotes of brightest group galaxies 
(BGGs) in fossils. A brief introduction to fossils is given in section 

2 where we also describe the sample and observations. Our analysis 
and the results are presented in section 3. A discussion and conclud- 
ing remarks are in section 4. We assume Ho = 70 km s _1 Mpc -1 
and n m = 0.3 with cosmological constant Qa = 0.7 throughout. 



2 FOSSIL GALAXY GROUPS 

In the class of galaxy groups known as "fossil groups", the group 
is dominated optically by a single luminous elliptical galaxy at 
the centre of extended luminous X-ray emission similar to that 
seen in bright X-ray groups. The X-ray emission in fossils is reg- 
ular and symmetric, indicating the absence of recent group merg- 
ing. The dominant giant elliptical galaxy has an optical luminos- 
ity similar t o BCGs. A cD galaxy has also been reported in a 
fossil group ( Mendes de Oliveira, Cvuriano & Sodre Jr. 2005). The 
observed properties of fossils, and their absence of L* galax- 
ies, suggest that they must be old gal axy groups. These prop- 
erties are discussed in recent studie s ( Khosrosh ahi et al. 1 1 2006 : 
Khosroshahi, Ponman & Jones] l2006h where we report a higher 
dark matter concentration in fossils, compared to non-fossils 
groups and clusters with similar masses, which is consistent with 
an early formation epoch. 

Observationall y a galaxy group is classified as a fossil if 
I Jones ^et al. I l2003h it has an X-ray luminosity of Lx,boi > 
10 42 /i^ 2 erg s _1 spatially extended to few 100 kpc, and the dom- 
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Figure 1. Ellipse fits to the surface brightness distribution of the central 
galaxy in fossil groups. All the images are from the Ks-band observations 
with the exception of the most distant fossil, RX J 1256.0+2556, which is 
observed in R-band. 5 arcsec scale bars are also shown. 



inant galaxy is at least 2 magnitudes brighter (in R-band) than the 
second ranked galaxy within half the projected virial radius of the 
group. The X-ray criterion guarantees the existence of a group size 
galaxy halo while the optical criterion assures that the M* galaxies 
are absent within the given radius which correspo nds to the radiu s 
for orbital decay by dynamical friction (Binn ev & Tremainell987F) . 
No upper limit is placed on the X-ray lumino sity or temperature, 
and r ecently a fossil galaxy cluster was found ( K hosroshahi et al. I 
l200d) . 



2.1 The sample 

This study makes use of a flux-limited sample of fossils found 
in the catalogue of spatially extended X-ray sources compiled by 
Wide Angle ROSAT Pointed Survey (WARPS) projec t. Details of 
the fo ssil identification and sample selection is given in ljones et al. I 
( 2003]). This is the largest statistical sample of fossil groups studied 
to date. In addition, the nearest known fossil group NGC 6482 and 
the first discovered fossil group, RX J 1340. 5+40 17, are included in 
our sample. The det ailed X-ray analysis of the sample is t he subject 
of a separate study (Khosroshah i. Ponman & Jonesl20 06). 
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2.2 Optical and near-IR observations 

The above sample was observed using the observational facilities 
of Issac-Newton Group of Telescopes (ING) and Kitt-peak Na- 
tional Observatory (KPNO). R-band images were obtained using 
the INT 2.5m wide field imager. Unfortunately the conditions were 
not photometric, and so further R-band imaging was obtained, in 
photometric conditions, with the 8k mosaic camera at the Univer- 
sity of Hawaii 2.2-m telescope, and in INT wide-field camera ser- 
vice time. The resultant photometric accuracy for all the systems is 
<0.05 mag. R-band observation of NGC 6482 was performed with 
the KPNO-0.9m in April 2005. Spectroscopic observations of the 
sample were also obtained, using slit spectroscopy on the KPNO 
4m, to exa mine the optical m embership of these groups. This is 
discussed in Khosroshahi, Ponman & Jones (2006). 

if s -band observations of the sample using UIST/UKIRT were 
performed in 2004. The seeing was measured to be ~ 1.0" . 
The data were reduced using the ORAC data reduction package 
(http://www.oracdr.0rg/1. Where multi-snaps were taken, the im- 
ages were co-added to increase the signal-to-noise. Figure [I] shows 
the K s images and their ellipse fits (section 3.3) for the central fos- 
sil galaxies in the sample with the exception of RX J1256.0+2556 
for which only the R-band data was available. 



3 ANALYSIS AND RESULTS 

Elliptical galaxies are usually single-component galaxies with ra- 
dial surface brightness profiles described by a de Vaucouleurs law 
(r 1 / 4 ). It has been shown that the Sersic profile (r 1//n ) gives a bet- 
ter fit, in general, for ellipticals with a wide luminosity range and 
in different environments (Trujillo et al. 2001; Khosroshahi et al. 
2004). Ellipticals are also divided based on their isophotal shapes. 
For this study we concentrate on the radial surface brightness pro- 
file, the ellipticity profile and the fourth order Fourier coefficient 
which is an indicator of boxy and disky isophotes. The anal- 
ysis is performed on both the R-band and K s -band images using 
the well-known IRAF/ellipse task. 



3.1 Radial surface brightness profile 

The Sersic profile, was used to model the R-band surface bright- 
ness distribution of the giant elliptical galaxies, using a two dimen- 
sional bulge/disk decomposition method. The values of n and the 
half-light radius, along with other parameters, are given in Table 
1. The analysis shows that the fossil central galaxies are best mod- 
elled with <n>=4.1±0.7. While this agrees in general with 
the surface brightness profiles of the remnants of collisionless disk 
mergers, these simulations are not able to produce gala xies as large 
as the dominant fossil galaxies JNaab & Bu rkert 2003). 

Our ground-based observations are inadequate for probing the 
radial surface brightness profiles within the central ~ 1 kpc, which 
is necessary for a power-law/core classification. As a result we limit 
our investigation to the Sersic fit to the galaxy. 



3.2 Ellipticity profile 

The ellipticity profiles presented in Fig |2| show a general pat- 
tern in which the ellipticity of the isophotes increases with the 
radius, with the exception of RX J1552.2+2013. Galaxies with 
high quality data show ellipticity increasing to 0.4-0.6 exceeding 
the ellipticity of the X-ray halo which is usually less than 0.3 
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Figure 2. Profiles of radial surface brightness, ellipticity and the fourth 
Fourier coefficient, B 4 , for RX Jl 119.7+2126, RX J1256.0+2556, RX 
J1331.5+1108, RX J1340.5+4017, RX J1416.4+2315, RX J1552.2+2013 
and NGC 6482. The profiles extracted from R-band and K s -bmd data are 
shown in dark (black) and grey (red), respectively. 



feuote & CanizareJ 19961) . The central galaxy in fossils are aligned 
with the underlying dark matter confirmed both in X-ray and lens- 
ing studie s. Similar alignment has be en noted for the BCGs in many 
clustersiFuller. West & Bridges! 19991). 



3.3 Isophotal analysis 

Fig 121 shows the results of the ellipse fits. In order to quan- 
tify the shape of the isophotes and to be able to make a direct 
comparison with similar analyses in the literature, we calculate 
q.4 / a, which is based on t he measured f ourth Fourier c oefficient, 
B 4 d-Torgensen et al.lll9QQh . Similarly to lBendeil Jl98Qh . a 4 /a = 
^Jdi/da * s q uan tifi e d at its peak value. In the absence of a peak 



the (14/cL is quantified at r e . Here e, a and / are the ellipticity, 
semi-major axis length and the surface brightness of the isophotes, 
respectively. As seen in Fig|2|none of the galaxies have predomi- 
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Table 1. Photometric properties of the sample galaxies. 



Group 


R.A. 


Dec. 


z 


Mr 




n 


r e 


kpc/arcsec 




(J2000) 


(J2000) 




mag 


X100 




arcsec 




RX Jl 119.7+2126 


11:19:43.6 


+21:26:51 


0.061 


-22.1 


0.1 


5.1 


8.9 


1.14 


RX J1256.0+2556 


12:56:03.4 


+25:56:48 


0.232 


-24.1 


0.1 


3.1 


7.3 


3.73 


RX J1331. 5+1108 


13:31:30.2 


+11:08:04 


0.081 


-22.9 


0.3 


4.3 


7.2 


1.53 


RX J1340.5+4017 a 


13:40:33.4 


+40:17:48 


0.171 


-23.0 


irr 


4.2 


7.6 


2.92 


RX J1416.4+2315 


14:16:26.9 


+23:15:32 


0.137 


-24.3 


0.7 


3.6 


12.1 


2.44 


RX J1552. 2+2013 


15:52:12.5 


+20:13:32 


0.135 


-24.0 


0.5 


4.6 


15.8 


2.40 


NGC 6482 6 


15:52:12.5 


+20:13:32 


0.013 


-22.9 


1.3 


3.8 


16.0 


0.26 



a This system is the first confirmed fossil group and not part of t he flux-limited sample of fossils. b This group is known to be the nearest fossil system 

i Khosroshahi, Jones & Ponman 2004 ) and is not part of the flux-limited sample. 
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Figure 3. The variation in isophotal shapes of early-type brightest group 
or cluster galaxies, and those in fossil groups (crosses), with the optical 
luminosity of the brightest galaxy. The comparison sample is a combination 
of earl y-type BGGs (triangles) and BCGs (circles) from Ellis &0' Sullivan 
(2006), for which a^/a values were available. 



nantly boxy isophotes, except in the outskirts were the statistics are 
very poor and the values of Ba are not well constrained. 

We compare the values of a^/a with those of the brightest 
galaxies in groups and clusters, BGGs and BCGs (Fig[3j. It is 
clear from this plot that none of the fossil's dominant galaxies have 
prominent boxy isophotes. Indeed some, including th e nearest fos- 
sil, NGC 6482 JKhosroshahL Jones & PonmaifeOO^. and the fos- 
sil cluster, RX J1416.4+2315 Ikhosrosh ahi et al.l200d). are highly 
disky isophote galaxies. An earlier study (Fa ber et al7l[l 997) gives 
an even higher value for the diskyness of NGC 6482. The com- 
paris on sample is a combinat ion of early-type BGGs and BCGs 
from Ellis &0'SuHivaiJ (2006) for which a^/a values were avail- 
able from earlier studies and therefore it is not a complete sample. 



4 DISCUSSION AND CONCLUSIONS 

This analysis shows that, despite apparent similarities, the domi- 
nant giant elliptical galaxy in fossil groups are different in their 
isophotal shapes, compared to the brightest central galaxies in 
non-fossil systems, especially in rich clusters. Luminous ellipti- 
cal galaxies in non-fossil groups and clusters do not present disky 
isophotes. Less luminous BGGs show disky and boxy isophotes 
in similar proportions. However, the observed frequency of boxy- 
isophote dominant fossil galaxies is apparently zero. 



If the central galaxies of fossil groups have indeed been 
formed from the merger of all major galaxies within the inner 
regions of the group, as we suppose, then some of these merg- 
ers would have been gas-rich, as group spirals are incorporated 
into the central merger-remnant. The disky character of central 
fossil galaxies is then consistent with the findings of numerical 
simulations, that disky i sophotes result from gas rich mergers. 
Khochfar & Burkert (2005) highlight the importance of the role of 
gas in galaxy mergers and show that the isophotal shapes of merger 
remnants are sensitive to the morphology of their progenitors and 
to subsequent gas infall. In contrast, boxy isophote ellipticals are 
formed by equal-mass mergers of bulge dominated galaxies. Such 
mergers are likely to occur at the core of clusters where most of the 
galaxies are gas poor. 

Almost 90% of BCGs studied bv lLaine etaDj2003h are core 
galaxies, ie. with flatter slope near the nucleus in their radial surface 
brightness profiles. The study by Faber et al. I ll997l) shows that a 
large fraction (~70%) of core galaxies have boxy isophotes, with as 
low as ~10% with disky isophotes, implying a strong association of 
boxy isophotes with core galaxies. Taking into account the conver- 
sion of cuspy cores into flat low -density cores by black hole merg- 
ing, [khochfar & Burkertl 120051) find that disky ellipticals should 
contain central density cusps whereas boxy ellipticals should in 
general be characterised by flat cores. Only rare low-luminosity 
boxy ellipticals, resulting from equal-mass mergers of disk galax- 
ies, could have power-law cores. Space resolution data is needed 
to study the core of the galaxies to verify the core and power- law 
property of dominant fossil galaxies. If future observations show 
that fossil group dominant galaxies are power-law galaxies, as ex- 
pected from their isophotes in light of the above argument, then 
they will be the first of such entities to grow to the size of BCGs. 

The link between fossil galaxy groups and the BCGs is further 
motivated by observational space density estimates of fo ssils which 
is fou nd to be 8% to 20% of X-ray luminous systems Ijones et ahl 
2003) and as large as the space density of poor and rich galaxy clus- 
ters combined. This means that there are enou gh fossils to provide 
BCG s to clusters. A recent theoretical study (Mil osavlievic et all 
2006) predicts 5%-40% of galaxy groups and l%-3% of galaxy 
clusters to be fossils. In the context of BCG formation, fossils ap- 
pear to be suitable environments for the formation of luminous gi- 
ant ellipticals around which clusters can form. This is a different 
formation mode to the one in which the BCG forms via mergers 
of brightest group galaxies during the cluster collapse. Numerical 
simulations suggest that the isophotes of the BCGs formed in the 
latter case will be predominantly boxy, characteristic of gas free 
("dry") mergers. 
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Figure 4. A comparison between the correlation of group/cluster veloc- 
ity dispersion and absolute B -magnitude of the dominant galaxy. The data 
points from Table 1 (except RX Jl 119.7+2126) are shown with filled cir- 
cl es. Diamonds re present two of the fossil candidate OLEGs, out of four, 
in lYoshioka et al. 1 120041) . The correlation is much tighter in fossils than 
in non-fossil groups (circles) and clusters (squares). The comparison non- 
fossil groups are GEMS X-ray selected grou ps with early-type BGG an d 
with group scale X-ray emission (G -sample in Osmond & Ponman 1 2004)). 
The cluster sample is selected from lGirardi et al. | ] 2002|) fo r whic h the lu- 
minosity of the central galaxy was available in Lin <! fcMohi (2004). 



The BGGs and BCGs with non-boxy isophotes in Fig|31could 
easily originate as fossil group central galaxies which have been 
incorporated into larger structures. The difference in photomet- 
ric structure seen clearly in many BCGs and BGGS, especially 
the most luminous ones, does not rule out the possibility that 
they originated in fossil groups. This could still be the case pro- 
vided that they have undergone later gas-free mergers within the 
cluster environment - for example, with the BGGs of inf ailing 
galaxy gr oups. About 40% of BCGs contain at least one secondary 
nucleus lLaine et al.1l2003l) . which strongly suggests the action of 
late mergers within the cluster environment. If fossils are indeed 
old and undisturbed systems then they should be found to have a 
very low incidence of multiple nuclei. 

In support of the above argument we show (Fig |4} a tight 
correlation between the luminosity of the central galaxy in fossils 
and the underlying gravitational mass probed by the group velocity 
dispersion. This shows that fossils form a homogeneous popula- 
tion in which the luminosity of the central galaxy is strongly tied 
to the property of its parent group. This property of the fossils is 
consistent with their early formation epoch and absence of recent 
merger. In contrast the large scatter in the distribution of the non- 
fossil galaxy groups and clusters, on the same plane, is understood 
to be merger driven. Absence of a recent major merger helps the 
dominant fossil galaxy preserve its original structure. 

We conclude that there is high chance that disky BCGs are 
formed in fossil groups. Boxy BCGs, could still result from progen- 
itor fossils, but would need to have undergone a dry merger within 
the cluster, probably as a result of merger with BGGs of inf ailing 
groups. 

We would like to thank Doug Burke, for his involvement in 
the UH 88" observations, and the INT for the service observations, 
Ewan O' Sullivan for his involvement in the KPNO observations of 
NGC 6482, Graham P. Smith for his valuable input which helped 
to improve the discussion, and Arif Babul for his useful comments. 
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